Sequential labeling with structural SVM under the F1 loss

Sequential labeling addresses the classification of sequential data and is of increasing importance for the classification and segmentation of video data. The model traditionally used for sequential labeling is the hidden Markov model where the sequence of class labels to be predicted is encoded as a Markov chain. In recent years, hidden Markov models and other structural models have benefited from minimum-loss training approaches which in many cases lead to greater classification accuracy. However, the loss functions available for training are restricted to decomposable cases such as the zero-one loss and the Hamming loss. Other useful losses such as the F1 loss, equal error rates and others are not available for sequential labeling. For this reason, in this paper we propose a training algorithm that can cater for the F1 loss and any other loss function based on the contingency table. Experimental results over the challenging TUM Kitchen Dataset depicting human actions in a kitchen scenario show that the proposed training approach leads to significant improvement of different performance metrics such as the classification accuracy (4.3 percentage points) and the F1 measure (8.9 percentage points).